Camera calibration method

ABSTRACT

Described is a method of calibrating a camera. The method comprises obtaining geographical coordinates of a selected physical point location within an image view of the camera and measuring an angle between an x-axis of a real-world coordinate system passing through said selected point location with respect to true north. The method includes using said obtained geographical coordinates, said measured angle, and projection data derived from characteristics of the camera to derive modified projection data for transforming a two-dimensional pixel coordinate system of the camera image view into a three-dimensional geographical coordinate system for point locations within the image view of the camera.

FIELD OF THE INVENTION

The invention relates to a camera calibration method to obtain absoluteposition coordinates for objects within an image view of the camera orin an image captured by the camera.

BACKGROUND OF THE INVENTION

Cameras are widely installed for performing different kinds of smartcity applications, for example, human/vehicle tracking, generatingtraffic statistics, vehicle speed detection, etc. Visual positioning isa technology that can provide geo-locations of objects in an area ofinterest by using image views or images captured by roadside cameras.Many applications, such as autonomous vehicle control and automaticvehicle parking, etc., can be further developed using detectedgeo-locations of objects in the area of interest. Increasing positioningaccuracy and decreasing cost is enhancing the demand for such visualpositioning. systems, especially in smart city environments. However,known camera calibration algorithms often exhibit one or more of thefollowing issues such as that the camera needs to have a movable fieldof view or that two or more cameras are required for coverage of a samearea. Other issues that may arise are that the outputted position dataor coordinates for observed objects are relative to a camera frame ofreference rather than absolute position coordinates such as globalposition geographical coordinates. Furthermore, the known calibrationmethods can be difficult to implement and time consuming with pooraccuracy in determining relative position coordinates for observedobjects.

WO2014/068073 discloses an optical measuring device for determiningthree-dimensional (3D) coordinates of an object. The measuring devicecomprises a projector device for illuminating: the object with at leastone predefined pattern; at least one camera for capturing atwo-dimensional (2D) image of the pattern as reflected from the object;computing means for measuring a sequence of brightness values of atleast one 2D image point from the 2D images; and calculating a 3Dcoordinate of an object point which is correlated with the measuredsequence of brightness values of the 2D image point. The outputted 3Dcoordinates are relative coordinates and more than one camera isrequired to determine 3D position coordinates for an observed object.

U.S. Pat. No. 9,679,382 describes a method of geo-referencing a firstimage of a scene acquired from a first imaging device based on at leastone second image of the scene acquired from a second imaging device. Themethod includes obtaining data indicative of an eligibility parameterfor one or more areas of the scene; selecting one or more pivot areasamong the one or more areas of the scene, wherein the eligibilityparameter of the pivot areas satisfy a. predefined criterion; for atleast some of the selected pivot areas, identifying tie points for thefirst and second images; and solving the external orientation of thefirst image using the identified tie points and a first imaging devicemodel. Once again, more than one camera is required for this system anda digital elevation model (DEM) map is required which is time consumingto obtain or to generate.

U.S. Pat. No. 8,934,008 discloses a method for determininggeo-location(s) in images. The calibration is performed by usingmultiple images captured from separate locations.

CN111461994 discloses a method to get pixel location of a referencepoint in an image and to obtain the absolute coordinate by using atransformation matrix based on global positioning system (GPS) data ofthe camera.

The publication entitled “Coordinate Rotation Derivations Rev ECoordinate Transformations via Euler Angle Rotations” by J. Riggspublished on 10 Apr. 2019 describes an example of a rotation matrix forrotating along an axis by an angle. It also discloses transformationfrom the North-East-Down (NED) coordinate system to the Earth-centredEarth fixed (ECEF) coordinate system.

The publication entitled “Stripmap SAR Data Collection Geometry on aRound Earth” published by Sandia. National Laboratories in December 2020describes an example of transformation from the ECEF coordinate systemto an absolute coordinate system, e.g., the World Geodetic System 1984(WGS84).

What is desired is an improved camera calibration method to obtainabsolute coordinates with high accuracy.

Objects of the Invention

An object of the invention is to mitigate or obviate to some degree oneor more problems associated with known methods of camera calibration fordetermining object positions.

The above object is met by the combination of features of the mainclaims; the sub-claims disclose further advantageous embodiments of theinvention.

Another object of the invention is to provide a novel camera system.

Another object of the invention is to provide a method of obtainingabsolute position coordinates for objects in an image view of a camera.

One skilled in the art will derive from the following description otherobjects of the invention. Therefore, the foregoing statements of objectare not exhaustive and serve merely to illustrate some of the manyobjects of the present invention.

SUMMARY OF THE INVENTION

The invention generally relates to a method of calibrating a camera andto a method of using image data from the camera to determine absoluteposition coordinates for objects within the camera image view. Themethod comprises obtaining geographical coordinates of a selectedphysical point location within an image view of the camera and measuringan angle between an x-axis of a real-world coordinate system passingthrough said selected point location with respect to true north. Themethod includes using said obtained geographical coordinates, saidmeasured angle, and projection data derived from characteristics of thecamera to derive modified projection data for transforming atwo-dimensional pixel coordinate system of the camera image view orimage data into a three-dimensional geographical coordinate system forpoint locations within the image view or image data of the camera. Thetransformation is one-to-one based on the assumption that all theobjects are lying on the Z=0 plane of the real-world coordinate system.

In a first main aspect, the invention provides a method of calibrating acamera comprising the steps of: obtaining geographical coordinates of aselected point location within an image view of the camera; measuring anangle between an x-axis of a real-world coordinate system with respectto true north, said x-axis passing through said selected point location;and using said obtained geographical coordinates, said measured angle,and projection data derived from characteristics of the camera to derivemodified projection data for transforming a two-dimensional pixelcoordinate system of the camera image view into a three-dimensionalgeographical coordinate system for point locations within the image viewof the camera.

In a second main aspect, the invention provides a camera comprising: amemory storing machine-readable instructions; and a processor forexecuting the machine-readable instructions such that, when theprocessor executes the machine-readable instructions, it configures thecamera to: receive data comprising geographical coordinates of aselected point location within an image view of the camera; receive datacomprising a measured angle between an x-axis of a real-world coordinatesystem with respect to true north, said x-axis passing through saidselected point location; and using said received data comprising thegeographical coordinates, said received data comprising the measuredangle, and projection data derived from characteristics of the camera toderive modified projection data for transforming a two-dimensional pixelcoordinate system of the camera image view into a three-dimensionalgeographical coordinate system for point locations within the image viewof the camera.

In a third main aspect, the invention provides a method of determininggeographical coordinates for a point location within an image view of acamera comprising the steps of: obtaining an image from the camera;selecting a point location in the image; obtaining two-dimensional pixelcoordinates for said selected point location in the image; andtransforming said two-dimensional pixel coordinates for said selectedpoint location in the image into three-dimensional geographicalcoordinates for said selected point location using projection datamapping a two-dimensional pixel coordinate system of the camera imageview into a three-dimensional geographical coordinate system.

The summary of the invention does not necessarily disclose all thefeatures essential for defining the invention; the invention may residein a sub-combination of the disclosed features.

The forgoing has outlined fairly broadly the features of the presentinvention in order that the detailed description of the invention whichfollows may be better understood. Additional features and advantages ofthe invention will be described hereinafter which form the subject ofthe claims of the invention. It will be appreciated by those skilled inthe art that the conception and specific embodiment disclosed may bereadily utilized as a basis for modifying or designing other structuresfor carrying out the same purposes of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing and further features of the present invention will beapparent from the following description of preferred embodiments whichare provided by way of example only in connection with the accompanyingfigures, of which:

FIG. 1 is a block schematic diagram of a smart city scene in whichcameras in accordance with the invention can implement the methods ofthe invention;

FIG. 2 is a schematic block diagram of an improved camera in accordancewith the invention;

FIG. 3 is a flow chart of set of methods in accordance with theinvention;

FIG. 4 illustrates the relationship between the camera 3D coordinatesystem and the camera 2D pixel coordinate system;

FIG. 5 illustrates the relationship between the 3D real-world coordinatesystem in the locality of the camera and the camera 3D coordinatesystem;

FIG. 6 illustrates the method of obtaining the camera's extrinsiccharacteristics;

FIG. 7 illustrates the method of obtaining the camera's intrinsiccharacteristics;

FIG. 8 illustrates the camera calibration method in accordance with theinvention; and

FIGS. 9 and 10 illustrate the calibration verification method inaccordance with the invention.

DESCRIPTION OF PREFERRED EMBODIMENTS

The following description is of preferred embodiments by way of exampleonly and without limitation to the combination of features necessary forcarrying the invention into effect.

Reference in this specification to “one embodiment” or “an embodiment”means that a particular feature, structure, or characteristic describedin connection with the embodiment is included in at least one embodimentof the invention. The appearances of the phrase “in one embodiment” invarious places in the specification are not necessarily all referring tothe same embodiment, nor are separate or alternative embodimentsmutually exclusive of other embodiments. Moreover, various features aredescribed which may be exhibited by some embodiments and not by others.Similarly, various requirements are described which may be requirementsfor some embodiments, but not other embodiments.

It should be understood that the elements shown in the FIGS, may beimplemented in various forms of hardware, software, or combinationsthereof. These elements may be implemented in a combination of hardwareand software on one or more appropriately programmed general-purposedevices, which may include a processor, a memory and input outputinterfaces.

The present description illustrates the principles of the presentinvention. It will thus be appreciated that those skilled in the artwill be able to devise various arrangements that, although notexplicitly described or shown herein, embody the principles of theinvention and are included within its spirit and scope.

Moreover, all statements herein reciting principles, aspects, andembodiments of the invention, as well as specific examples thereof, areintended to encompass both structural and functional equivalentsthereof. Additionally, it is intended that such equivalents include bothcurrently known equivalents as well as equivalents developed in thefuture, i.e. any elements developed that perform the same function,regardless of structure.

Thus, for example, it will be appreciated by those skilled in the artthat the block diagrams presented herein represent conceptual views ofsystems and devices embodying the principles of the invention.

The functions of the various elements shown in the figures may beprovided through the use of dedicated hardware as well as hardwarecapable of executing software in association with appropriate software.When provided by a processor, the functions may be provided by a singlededicated processor, by a single shared processor, or by a plurality ofindividual processors, some of which may be shared. Moreover, explicituse of the term “processor” or “controller” should not be construed torefer exclusively to hardware capable of executing software, and mayimplicitly include, without limitation, digital signal processor (“DSP”)hardware, read-only memory (“ROM”) for storing software, random accessmemory (“RAM”), and non-volatile storage.

In the claims hereof, any element expressed as a means for performing aspecified function is intended to encompass any way of performing thatfunction including, for example, a) a combination of circuit elementsthat performs that function or b) software in any form, including,therefore, firmware, microcode, or the like, combined with appropriatecircuitry for executing that software to perform the function. Theinvention as defined by such claims resides in the fact that thefunctionalities provided by the various recited means are combined andbrought together in the manner which the claims call for. It is thusregarded that any means that can provide those functionalities areequivalent to those shown herein.

One of the most common sensors used in smart city developments aresurveillance cameras. Millions of such cameras are being installed incities and towns worldwide to perform functions such as human/vehicletracking and positioning, speed detection, providing data to vehiclesand pedestrians though suitably enabled electronic devices, providingdata to central control systems, etc. Such surveillance cameras can beusefully used to assist in traffic control and policing, pedestriansafely, and automatic vehicle parking to identify but a few of theirfunctions. However, some of these functions require the vehicle and/orpedestrian position data to be highly accurate within a frame ofreference. In many existing systems, the camera relative frame ofreference providing position data for vehicles and/or pedestrians whichare relative to the position of the camera rather than absolute positiondata such as global position system geographical coordinates. Theusefulness of such surveillance cameras is greatly enhanced if they areadapted to provide absolute position data for vehicles and/orpedestrians or for any objects falling within the view of the camera.

One of the difficulties in achieving the aforesaid objective is thedifficulty of calibrating such a camera to provide absolute positiondata for objects in the camera image view.

In one of its main aspects, the present invention provides a novel andsimple method for calibrating the relationship between a real-worldcoordinate system in the locality of the camera with an absolutecoordinate system such as a three-dimensional geographical coordinatesystem. This in turn enables a 2D pixel coordinate system of the cameraimage view to be calibrated to three-dimensional geographical coordinatesystem, an absolute positioning system.

The method may include calibrating one or both of the intrinsiccharacteristics of the camera and the extrinsic characteristics of thecamera if these are not already known.

Furthermore, the method may include verifying the calibration resultsusing a novel and simple method requiring nothing more than the sametools used for the main calibration step of the method.

The main aim of the method of the first main aspect of the invention isto transform the 2D pixel coordinate system of the camera image viewinto the 3D geographical coordinate system for any selected pointlocations lying in a plane such as the ground within the image view ofthe camera.

FIG. 1 comprises a smart city scene 100 in which one or more cameras 101are suitably enabled in accordance with the invention to capture andprovide image data from which absolute position data, e.g., globalpositioning system (GPS) data or the like, for objects 102 within thecamera views can be obtained. Each camera 101 has a specified cameraview, e.g., field of view (FOV) of a part of the city scene 100, whichin this instance comprises a highway 103 with a pedestrian crossing 104.

In the example of FIG. 1 , the cameras 101 are mounted on street-lamps105 but it will be understood that the cameras 101 could be mounted onany suitable structures including buildings, etc. The camera views mayoverlap although this is not essential, but camera overlapping areas areuseful for cross camera object identification and tracking. Each camera101 is adapted to obtain image data of its respective camera image viewfrom which absolute position data can be obtained for the objects 102located within the camera's image view. Each camera 101 may be enabledwith object recognition software to enable it to recognize and classifyobjects 102 within its camera view but that is not the focus of thepresent invention. Each camera 101 is preferably enabled to select apoint location on each object 102 lying in a plane, for example, theground within its camera view for which absolute position data can beobtained from the camera image data. Preferably, point locations areselected at the base of an end of an object 102 nearest the camera 101and centrally of said object's base, the absolute position data for saidselected point location represents the absolute location of the object102.

Each camera 101 may be enabled to determine by itself the absoluteposition data for each object 102 within its camera view and to reportsaid data to other entities in the smart city system such as reportingsaid data to vehicle based electronic devices, pedestrian handheldelectronic devices, other system sensors including other cameras 101,system gateways (not shown) and one or more system central controllers106. Additionally, or alternatively, the cameras 101 may each be linkedto a system server 107 and a database 108 by a suitable communicationnetwork 109 such as an Ethernet network with the cameras 101 beingenabled to provide camera image data via, for example, real-time encodedvideo streams using a video compression method such as the known videocompression standard for high-definition digital video identified as“H.264” to the server 107 and/or the database 108. The server 107 may beconfigured to determine the absolute position data for objects 102within the camera views from the camera image data provided by thevarious cameras 101, to store said absolute position data in thedatabase 108, and to communicate said absolute position data as requiredto the other smart city system entities.

FIG. 2 shows an exemplary embodiment of an improved camera 101 inaccordance with concepts of the present invention. In the illustratedembodiment, the camera 101 may include communication equipment 110 suchas a network node, a network card, or a network circuit, etc. operatingin, for example, a 5G communications system environment, although theimproved camera 101 of the invention is not limited to operating in a 5Gcommunications system but could include communication equipment 110 foroperating in any suitable communications network 109.

The camera 101 may comprise a plurality of functional blocks forperforming various functions thereof. For example, the camera 101 mayinclude a transceiver module 111 for transmitting and receiving datasignals from other smart city system entities. The transceiver module111 provides signal processing and is configured to provide signalsand/or information extracted therefrom to functional block module(s) 112such as may comprise various data sink, control element(s), userinterface(s), etc. Irrespective of the particular configuration of thetransceiver module 111, embodiments include data processing, module 113which may be disposed in association with the transceiver module 111 forfacilitating accurate processing of data in accordance with theinvention.

Although the data processing module 113 is shown as being deployed aspart of the transceiver module 111 (e.g., comprising a portion of thetransceiver module control and logic circuits), there is no limitationto such a deployment configuration according to the concepts of theinvention. For example, the data processing module 113 may be deployedas a separate functional block of the camera 101 that is distinct from,but connected to, the transceiver module 111. The data processing module113 may for example, be implemented using logic circuits and/orexecutable code/machine readable instructions stored in a memory 114 ofthe camera 101 for execution by a processor 115 to thereby performfunctions as described herein. For example, the executable code/machinereadable instructions may be stored in one or more memories 114 (e.g.,random access memory (RAM), read only memory (ROM), flash memory,magnetic memory, optical memory, or the like) suitable for storing oneor more instruction sets (e.g., application software, firmware,operating system, applets, and/or the like), data (e.g., configurationparameters, operating parameters and/or thresholds, collected data,processed data, and/or the like), etc. The one or more memories 114 maycomprise processor-readable memories for use with respect to one or moreprocessors 115 operable to execute code segments of data processingmodule 113 and/or utilize data provided thereby to perform functions ofthe data processing module 113 as described herein. Additionally, oralternatively, the data processing module 113 may comprise one or morespecial purpose processors (e.g., application specific integratedcircuit (ASIC), field programmable gate array (FPGA), graphicsprocessing unit (GPU), and/or the like configured to perform functionsof the data processing module 113 as described herein.

FIG. 3 is a flow chart of a set of methods 120 implemented by the dataprocessing module 113 of the camera 101 in accordance with a first mainaspect of the invention. The set of methods 120 comprises a firstoptional method 121 of calibrating the intrinsic characteristics of thecamera 101. This method 121 calibrates the intrinsic characteristics ofthe camera 101 comprising one or more of the camera lens' focal length,the camera optical center, and optionally the coefficients of radial andtangential distortion of the camera lens, from which can be obtained aprojective transformation of the camera 3D coordinate system to thecamera 2D pixel coordinate system for the camera image view or capturedimages.

The set of methods 120 comprises a second optional method 122 ofcalibrating the extrinsic characteristics of the camera 101. This method122 calibrates the extrinsic characteristics of the camera 101 bycalibrating the 3D real-world coordinate system in the locality of thecamera 101 with the 3D camera coordinate system.

The main non-optional method 123 of the invention comprises calibratingthe relationship between the 3D real-world coordinate system in thelocality of the camera 101 with an absolute 3D coordinate system as willbe described below.

A final optional method 124 of the set of methods 120 comprises takingabsolute position coordinates for a multiplicity of test points in thelocality of the camera 101, i.e., within the camera image view, andcomparing the absolute position coordinates to absolute position datacalculated for corresponding point locations in image data captured bythe camera 101.

FIG. 4 illustrates the relationship between the camera 3D coordinatesystem 130 and the camera 2D pixel coordinate system 131. The mainintrinsic characteristics of the camera 101 comprise the focal length ofthe camera lens in pixel units anal the optical center of the cameraalso in pixel units. The camera intrinsic characteristics can berepresented by a 3 by 3 matrix K:

$K = \begin{bmatrix}f_{x} & 0 & c_{x} \\0 & f_{y} & c_{y} \\0 & 0 & 1\end{bmatrix}$

where

[c_(x), c_(y)] is the optical center (the principal point) in pixels;and

[ƒ_(x), ƒ_(y)] is the lens focal length in pixels

The matrix K represents a projective transformation from the camera 3Dcoordinate system 130 to the camera 2D pixel coordinate system 131 andvice-versa. The matrix K, however, does not account for lens distortionso optionally calibrating the camera intrinsic characteristics mayinclude adding distortion correction coefficients for one or both ofradial distortion and tangential distortion. Radial distortion occurswhen light rays bend more near the edges of the lens than at the opticalcenter of the lens. Tangential distortion occurs when the lens and theimage plane are not parallel. Both the radial distortion and thetangential distortion are non-linear. The relationships for correctingradial distortion comprise:x _(distorted) =x(1+k ₁ *r ² +k ₂ r ⁴ +k ₃ *r ⁶)y _(distorted) =y(1+k ₁ *r ² +k ₂ r ⁴ +k ₃ *r ⁶)

where x, y are the undistorted pixel locations;

k₁, k₂, k₃ are the radial distortion coefficients of the camera lens;and

r²:x²+y².

The relationships for correcting tangential distortion comprise:x _(distorted) =x+[2*p ₁ *x*y+p ₂*(r ²+2*x ²)]y _(distorted) =y+[p ₁*(r ²+2*y ²)+2*p ₂ *x*y]

where x, y are the undistorted pixel locations,

p₁, p₂ are the tangential distortion coefficients of the camera lens;and

r²: x²+y².

FIG. 5 illustrates the relationship between the real-world coordinatesystem 132 in the locality of the camera 101 and the camera 3Dcoordinate system 130. The camera extrinsic relationship comprises a 3by 3 rotation matrix R and a 3 by 1 translation vector T. The origin ofthe camera's 3D coordinate system 130 is its optical center and itsx-axis and its y-axis define the camera image plane. The 3 by 3 rotationmatrix R and a 3 by 1 translation vector T are represented by thefollowing matrix:

$\begin{matrix}\begin{bmatrix}r_{11} & r_{12} & r_{13} & t_{1} \\r_{21} & r_{22} & r_{23} & t_{2} \\r_{31} & r_{32} & r_{33} & t_{3}\end{bmatrix} \\\begin{matrix}\lbrack R\rbrack & \lbrack T\rbrack\end{matrix}\end{matrix}.$

The camera intrinsic characteristics and the camera extrinsiccharacteristics can be combined to provide projection data or aprojection matrix P to transform the real-world coordinate system 132 inthe locality of the camera 101 to the camera 2D pixel coordinate system131:

$\underset{\underset{Coordinates}{2D{Image}}}{\begin{bmatrix}u \\v \\1\end{bmatrix}} = {\underset{\underset{({{{Optical}{Centre}},{scaling}})}{{Intrinsic}{properties}}}{\begin{bmatrix}f_{x} & 0 & c_{x} \\0 & f_{y} & c_{y} \\0 & 0 & 1\end{bmatrix}}\underset{\underset{\underset{{{and}{translation}})}{({{Camera}{Rotation}}}}{{Extrinsic}{properties}}}{\begin{bmatrix}r_{11} & r_{12} & r_{13} & t_{1} \\r_{21} & r_{22} & r_{23} & t_{2} \\r_{31} & r_{32} & r_{33} & t_{3}\end{bmatrix}}\underset{\underset{Coordinates}{3D{World}}}{\begin{bmatrix}X \\Y \\Z \\1\end{bmatrix}}}$

The above camera projection matrix P does not include distortioncorrection, but distortion correction can be compensated as follows:

$\begin{matrix}{\begin{bmatrix}x \\y \\z\end{bmatrix} = {{R\begin{bmatrix}X \\Y \\Z\end{bmatrix}} + t}} \\{x^{\prime} = {x/z}} \\{y^{\prime} = {y/z}} \\{x^{\prime\prime} = {{x^{\prime}\frac{1 + {k_{1}r^{2}} + {k_{2}r^{4}} + {k_{3}r^{6}}}{1 + {k_{4}r^{2}} + {k_{5}r^{4}} + {k_{6}r^{6}}}} + {2p_{1}x^{\prime}y^{\prime}} + {p_{2}\left( {r^{2} + {2x^{\prime 2}}} \right)}}} \\{y^{\prime\prime} = {{y^{\prime}\frac{1 + {k_{1}r^{2}} + {k_{2}r^{4}} + {k_{3}r^{6}}}{1 + {k_{4}r^{2}} + {k_{5}r^{4}} + {k_{6}r^{6}}}} + {p_{1}\left( {r^{2} + {2y^{\prime 2}}} \right)} + {2p_{2}x^{\prime}y^{\prime}}}} \\{{{where}r^{2}} = {x^{\prime 2} + y^{\prime 2}}} \\{u = {{f_{x} \star x^{\prime\prime}} + c_{x}}} \\{v = {{f_{y} \star y^{\prime\prime}} + c_{y}}}\end{matrix}$

where x″ and y″ are the compensated camera image plane x and y values.

In the following description, it will be assumed that there is no lensdistortion in the camera 101.

A preferred method 122 of determining the camera extrinsiccharacteristics uses a first patterned planar device 140 of known sizeplaced preferably entirely within the image view of the camera 101 andpreferably appearing in the image as large as possible. The method 122determines the position and orientation of the camera 101 preferablywith respect to a designated origin of the first patterned planar device140 denoted as “World center (0,0,0)” in FIG. 8 . The first patternedplanar device 140 preferably comprises a first planar chessboardpatterned device 140 which defines the 3D real-world coordinate system132 in the locality of the camera 101 with axis Z=0. However, thechessboard pattern of the first planar chessboard patterned device 140is only meaningful locally to the camera 101 and cannot be directly usedto obtain absolute position data for selected points in the camera imageview.

The preferred method 122 of determining the camera extrinsiccharacteristics assumes that the same camera 101 is also used forseparately determining the camera's intrinsic characteristics asdescribed below. The method 122, as illustrated in FIG. 6 , comprisesplacing the first planar patterned chessboard device 140 on the groundwithin the camera's FOV, i.e., within the camera's image view, such thatthe whole first planar chessboard patterned device 140 is within thecamera's FOV. The first planar chessboard patterned device 140 should beplaced flat on the ground. Preferably, the first planar chessboardpatterned device 140 has a size which is at least 150 pixels by 150pixels in the camera's image view. This would typically be about 1.5 mby 1.5 m in a typical smart city surveillance camera scene 100. Themethod 122 includes taking at least one image of the first planarchessboard patterned device 140 with the camera 101. The elevation angleof the camera 101 is dependent on the region of interest (ROI) of thecamera 101 for detection of objects 102 located within the camera's FOV.The elevation angle may be about 30°. The camera 101 may provide thecaptured image to the server 107 and/or the database 108. The overallsquare size in meters of the first planar chessboard patterned device140 and its number of rows and columns are inputted to the camera 101and/or to the server 107. The chessboard dimensions may have alreadybeen uploaded to the camera 101 and/or the server/database 107/108before the imaging process of the first planar chessboard patterneddevice 140 is performed to thereby time it extrinsic characteristicscalibration method 122. In any event, the extrinsic characteristicsmatrix

$\begin{bmatrix}R \\t\end{bmatrix}$is determined from the captured image data and the dimensions data ofthe first planar chessboard patterned device 140 using any suitablecamera extrinsic calibration method. A suitable tool for implementingthe camera extrinsic calibration method is known as “OpenCV™” whichcomprises the publicly available Open Source Computer Vision Library.The OpenCV™ SolvePnP function may be used. The resulting extrinsiccharacteristics matrix

$\begin{bmatrix}R \\t\end{bmatrix}$is saved to the camera 101 and/or the server/database 107/108.

The preferred method 121 of determining the camera intrinsiccharacteristics, as illustrated in FIG. 7 , involves taking multipleimages of a second planar patterned device 150 of smaller size than thefirst planar patterned device 140. The method may involve printing thesecond planar patterned device 150 comprising a second planar chessboardpatterned device 150 on, for example, A4 sized paper or card. The methodpreferably involves taking multiple images of the second planarchessboard patterned device 150 with the camera 101 by placing carholding said second planar chessboard patterned device 150 in thecamera's FOV at multiple different viewing angles and/or at multipledifferent distances from the camera 101. Preferably, as many as 20 to 30images are obtained. The dimensions of the printed second planarchessboard patterned device 150 are stored in the camera 101 and/or theserver/database 107/108 and this may be done in advance of the imagingstep. The intrinsic characteristics matrix K is determined from acombination of the captured image data and the dimensions data of thesecond planar chessboard patterned device 150 using any suitable cameraintrinsic calibration method. A suitable tool for implementing thecamera intrinsic calibration method 121 is the OpenCV™ calibrate camerafunction. The resulting intrinsic characteristics matrix K is saved tothe camera 101 and/or the server/database 107/108.

The preferred method 122 of determining the camera extrinsiccharacteristics and the preferred method 121 of determining the cameraintrinsic characteristics are, as indicated above, optional methods ifthe intrinsic and extrinsic characteristics of the camera 101 are notalready known. Furthermore, any suitable methods for obtaining theintrinsic and extrinsic characteristics of the camera 101 could beimplemented.

The key, non-optional method 123 in accordance with the invention is themethod of modifying the projection matrix P to make it suitable fortransforming the 2D pixel coordinate system 131 of the camera image viewto a 3D global geographical coordinate system for point locations withinthe image view of the camera, i.e., to an absolute positioning system orabsolute coordinate system comprising a one-to-one transformation formapping to the Z=0 plane of the 3D real-world coordinate system.

The method 123 should use the same first planar chessboard patterneddevice 140 as used in the method 122 for determining the cameraextrinsic characteristics. The measurements for the method 123 aresimple and can be completed in a short time of typically less than oneminute. The method 123 comprises measuring absolute coordinates for aselected point location, i.e., the origin of the chessboard coordinatesystem, within the image view of the camera 101. The selected pointlocation should be a selected corner point 141 on the first planarchessboard patterned device 140 and is preferably an internal cornerpoint 141 on said planar chessboard patterned device 140, i.e., not aphysical external corner point of the device 140 but a corner point 141in the chessboard pattern of the device 140 as exemplified by FIG. 8 .The selected corner point 141 can be considered as the ‘world center’ ororigin point [0, 0, 0] for the 3D real-world coordinate system in thelocality of the camera 101. The absolute coordinates, i.e., the globalgeographical coordinates, of the origin point [0, 0, 0] may be measuredusing any suitable global position device such as a real-time kinematic(RTK) device.

In addition to obtaining the absolute coordinates of the origin point[0, 0, 0], the method 123 includes measuring an angle a between anx-axis 142 of the real-world coordinate system with respect to truenorth 144, said x-axis 142 passing through said origin point [0, 0, 0].Preferably, the method 123 involves measuring an anti-clockwise angle αwith respect to true north. The angle α may be measured using anysuitable compass device such as an e-compass device in which the anglemeasurement error is preferably less than 0.5 degree.

The method 123 further comprises using said obtained absolutecoordinates of the origin point [0, 0, 0], the measured angle α, andprojection data derived from the intrinsic and extrinsic characteristicsof the camera 101 to derive modified projection data for transformingthe 2D pixel coordinate system 131 of the camera image view into the 3Dgeographical coordinate system to enable global geographical coordinatesto be derived for any selected point location, i.e., pixel position,within the image view of the camera 101 or within an image captured bythe camera 101.

The method 123 thereby changes the earlier described camera projectionmatrix P into a modified or integrated projection matrix P′ whichenables pixel positions in the 2D pixel coordinate system 131 of thecamera 101 to be transformed or converted into global geocentric orgeographical coordinates in the 3D absolute position system.

The method 123 uses a first rotation matrix based on the measured angleα and a second rotation matrix based on the measured latitude andlongitude geographical coordinates of the origin point [0, 0, 0 ] tomodify the camera projection matrix P to obtain the integratedprojection matrix P′.

The camera projection matrix P as shown again below enables a 3Dreal-world coordinate [X Y Z], namely the coordinate system of the firstplanar patterned chessboard device 140, to be converted to a 2D pixelposition [u, v] on the camera image view or image data in the localityof the camera 101.

Camera projection matrix P (assuming no camera lens distortion)comprises:

${s\begin{bmatrix}u \\v \\1\end{bmatrix}} = {{\begin{bmatrix}f_{x} & 0 & c_{x} \\0 & f_{y} & c_{y} \\0 & 0 & 1\end{bmatrix}\begin{bmatrix}r_{11} & r_{12} & r_{13} & t_{1} \\r_{21} & r_{22} & r_{23} & t_{2} \\r_{31} & r_{32} & r_{33} & t_{3}\end{bmatrix}}\begin{bmatrix}X \\Y \\Z \\1\end{bmatrix}}$

When the Z value in the 3D real-world coordinate system in the localityof the camera 101 is taken as zero, there is a one-to-one relationshipor solution from pixel positions in the camera image view or image datato the 3D real-world coordinate system in the locality of the camera101.

With Z set to zero and expanding the 3^(rd) row of the camera projectionmatrix P, this can be expressed as:S=r ₁₃ *t ₁ +r ₂₃ *t ₂ +r ₃₃ *t ₃ /r ₁₃ *u+r ₂₃ *v+r ₃₃*1.0

This expression cab be rewritten as:

${s\begin{pmatrix}u \\v \\1\end{pmatrix}} = {{{{KR}\begin{pmatrix}X \\Y \\0\end{pmatrix}} + {T{{KR}^{- 1}\left\lbrack {{s\begin{pmatrix}u \\v \\1\end{pmatrix}} - T} \right\rbrack}}} = \begin{pmatrix}X \\Y \\0\end{pmatrix}}$

such that a unique X, Y position in the 3D real-world coordinate systemcan be solved for a 2D pixel position [u, v] on the camera image view orimage data.

The camera projection matrix P is then modified in method 123 to theintegrated projection matrix P′ by the first and second rotationmatrices as follows:

$\begin{matrix}{\begin{pmatrix}{XE} \\{YE} \\{ZE}\end{pmatrix} = {{\begin{bmatrix}{{- {\sin(\phi)}}{\cos(\lambda)}} & {- {\sin(\lambda)}} & {{- {\cos(\phi)}}\cos(\lambda)} \\{{- \sin}(\phi){\sin(\lambda)}} & {\cos(\lambda)} & {{- {\cos(\phi)}}{\sin(\lambda)}} \\{\cos(\phi)} & 0 & {- {\sin(\phi)}}\end{bmatrix}\begin{pmatrix}{\cos(a)} & {\sin(a)} & 0 \\{- {\sin(a)}} & {\cos(a)} & 0 \\0 & 0 & 1\end{pmatrix}\begin{pmatrix}{XC} \\{YC} \\{ZC}\end{pmatrix}} + {PRef}}} \\\begin{matrix}{{First}{rotation}{matrix}} & {{Second}{rotation}{matrix}}\end{matrix}\end{matrix}$

where [XE, YE, ZE] represents the 3D absolute coordinate system (globalgeocentric or geographical coordinate system), [XC, YC, ZC] representsthe 3D real-world coordinate system in the locality of the camera 101and PRef is the absolute coordinate position of the origin point [0, 0,0].

As:

$\begin{pmatrix}{XC} \\{YC} \\0\end{pmatrix} = {{KR}^{- 1}\left\lbrack {{s\begin{pmatrix}u \\v \\1\end{pmatrix}} - T} \right\rbrack}$

then the integrated projection matrix P′ can be expressed as:

$\begin{pmatrix}{XE} \\{YE} \\{ZE}\end{pmatrix} = {{\begin{bmatrix}{{- {\sin(\phi)}}{\cos(\lambda)}} & {- {\sin(\lambda)}} & {{- {\cos(\phi)}}\cos(\lambda)} \\{{- \sin}(\phi){\sin(\lambda)}} & {\cos(\lambda)} & {{- {\cos(\phi)}}{\sin(\lambda)}} \\{\cos(\phi)} & 0 & {- {\sin(\phi)}}\end{bmatrix}\begin{pmatrix}{\cos(a)} & {\sin(a)} & 0 \\{- {\sin(a)}} & {\cos(a)} & 0 \\0 & 0 & 1\end{pmatrix}{{KR}^{- 1}\left\lbrack {{s\begin{pmatrix}u \\v \\1\end{pmatrix}} - T} \right\rbrack}} + {PRef}}$

The integrated projection matrix P′ makes it possible to find 3Dabsolute coordinates for any 2D pixel position [u, v] on the cameraimage view or image data system in the locality of the camera.

in the foregoing, the 3D absolute coordinates may be geocentriccoordinates according to the Earth-centered, Earth-fixed (ECEF)coordinate system as will be more fully described below, but the method123 preferably includes a final transformation of the ECEF coordinatesystem to a truly global geographical coordinate system such as theWorld Geodetic System 1984 (WGS84), although any absolute geographicalcoordinate system can be used.

As described above, the camera projection matrix P enables a 2D pixelposition [u, v] on the camera image view or image data to be convertedto the 3D real-world coordinate system in the locality of the camera101, namely the coordinate system of the first planar patternedchessboard device 140 when the constraint Z=0 is applied. The method 123modifies the camera projection matrix P to provide an integrated cameraprojection matrix P′ to enable a 2D pixel position [u, v] on the cameraimage view or image data to be converted to a 3D global geocentric orgeographical position system, i.e., an absolute position system.

The method 123 preferably comprises firstly transforming the 3Dreal-world coordinate system in the locality of the camera 101, namelythe coordinate system of the first planar patterned chessboard device140, to an intermediary coordinate system such as the North-East-Down(NED) coordinate system, The NED coordinate system is the nearestcoordinate system to that of the first planar patterned chessboarddevice 140 since they both occupy a tangential plane to the earth. Thesetwo coordinate systems differ by just a rotation in the Z-axis (downaxis). Consequently, converting or transforming the coordinate system ofthe first planar patterned chessboard device 140 to the NED coordinatesystem makes it easier to then convert or transform to other absolutecoordinate systems such as the ECEF geocentric coordinate system and/orthe WGS84 global geographical coordinate system.

The method 123 preferably also comprises transforming the intermediarycoordinate system, e.g., the NED coordinate system to the ECEFcoordinate system. This is because the NED coordinate system is arelative coordinate system whereas ECEF is an absolute Euclideancoordinate system which has been found to be a good stepping-stone forfurther converting to a commonly used global geographical coordinatesystem such as WGS84. WGS84 is preferred as it is the most commonly usedglobal geographical coordinate system.

The NED coordinate system is a right-handed local tangent planecoordinate system on the Earth's surface. In the NED coordinate system,the normal, i.e., “Down” points to the Earth's center and north pointsto the true North/geodetic north, namely the direction along the Earth'ssurface towards the geographic North Pole. For a flat small area such asthat of the first planar patterned chessboard device 140, the Earth'ssurface can be considered as planar or horizontal from a local point ofview. Consequently, at this scale, the tangent plane is essentially thesame as the true surface of the Earth. In view of this, the first planarpatterned chessboard device 140 comprises a local tangent plane to theEarth's surface. The conversion or transformation of the 3D real-worldcoordinate system in the locality of the camera 101 to the NEDcoordinate system comprises a rotation transformation on the Z-axis(Down-axis) based on the angular difference between the x-axis 142 andtrue north 144. In other words, conversion or transformation of the 3Dreal-world coordinate system in the locality of the camera 101 to theNED coordinate system comprises a rotation transformation derived fromthe measured angle α which provides the first rotation matrix referredto in the foregoing. The first rotation matrix is given by:

$\begin{pmatrix}{\cos(a)} & {\sin(a)} & 0 \\{- {\sin(a)}} & {\cos(a)} & 0 \\0 & 0 & 1\end{pmatrix}$

The method 123 preferably also comprises transforming the NED coordinatesystem to the ECU coordinate system The NED coordinate system and theECEF coordinate system are Euclidean coordinate systems which differ bya rigid transform comprising both rotation and translation.Consequently, converting from the NED coordinate system to the ECEFcoordinate system is based on the following relationship:

${P_{ECEF} = {{RP_{NED}} + P_{REF}}}{R = \begin{bmatrix}{{- {\sin(\phi)}}{\cos(\lambda)}} & {- {\sin(\lambda)}} & {{- {\cos(\phi)}}\cos(\lambda)} \\{{- \sin}(\phi){\sin(\lambda)}} & {\cos(\lambda)} & {{- {\cos(\phi)}}{\sin(\lambda)}} \\{\cos(\phi)} & 0 & {- {\sin(\phi)}}\end{bmatrix}}$

where P_(NED) is a 3D position in the NED system, P_(ECEF) is thecorresponding ECEF position, P_(Ref) is the reference ECEF position,i.e., the ECEF coordinate, position of the origin point [0, 0, 0] of theof the first planar patterned chessboard device 140, and R is the secondrotation matrix whose columns comprise “north”, “east”, and “down” axes.R is the matrix of the measured latitude ϕ and longitude λ geographicalcoordinates for P_(Ref).

The transformation from the 3D real-world coordinate system in thelocality of the camera 101 [XC, YC, ZC] to the ECEF coordinate system(XE, YE, ZE) can be integrated into the following formula:

$\begin{pmatrix}{XE} \\{YE} \\{ZE}\end{pmatrix} = {{\begin{bmatrix}{{- {\sin(\phi)}}{\cos(\lambda)}} & {- {\sin(\lambda)}} & {{- {\cos(\phi)}}\cos(\lambda)} \\{{- \sin}(\phi){\sin(\lambda)}} & {\cos(\lambda)} & {{- {\cos(\phi)}}{\sin(\lambda)}} \\{\cos(\phi)} & 0 & {- {\sin(\phi)}}\end{bmatrix}\begin{pmatrix}{\cos(a)} & {\sin(a)} & 0 \\{- {\sin(a)}} & {\cos(a)} & 0 \\0 & 0 & 1\end{pmatrix}\begin{pmatrix}{XC} \\{YC} \\{ZC}\end{pmatrix}} + T}$

where α is the measured angle between the x-axis 142 of the 3Dreal-world coordinate system and true north 144 in an anti-clockwisedirection; ϕ and λ are the latitude and longitude geographicalcoordinates measured by the RTK device at the origin point [0, 0, 0] ofthe first planar patterned chessboard device 140; and T=P_(Ref) whichcomprises the ECEF coordinates of the origin point [0, 0, 0].

In practice, it has been found to be easier to measure T using a WGS84device and then converting the obtained global geographical coordinatesto ECEF coordinates.

Finally, the method 123 preferably includes converting or transformingthe ECEF coordinate system to preferably a widely used global coordinatesystem such as WGS84. The ECEF coordinate system is a Euclideancoordinate system with a one-to-one non-linear mapping with WGS84, whichis a polar coordinate system. Conversion from the ECEF coordinate systemto WGS84 may be calculated by the closed form algorithm published byJiJie Zhu in 1993. Third party software like MATLAB® or pymap3d providea suitable API for the conversion. The integrated camera projectionmatrix P′ can therefore be further modified to include the conversionfrom the ECEF coordinate system to WGS84 in accordance with thefollowing:

$\begin{pmatrix}{Lat} \\{Lon} \\{Alt}\end{pmatrix} = {{eccef\_ to}{\_ wgs}84\left( {{\begin{bmatrix}{{- {\sin(\phi)}}{\cos(\lambda)}} & {- {\sin(\lambda)}} & {{- {\cos(\phi)}}\cos(\lambda)} \\{{- \sin}(\phi){\sin(\lambda)}} & {\cos(\lambda)} & {{- {\cos(\phi)}}{\sin(\lambda)}} \\{\cos(\phi)} & 0 & {- {\sin(\phi)}}\end{bmatrix}\begin{pmatrix}{\cos(a)} & {\sin(a)} & 0 \\{- {\sin(a)}} & {\cos(a)} & 0 \\0 & 0 & 1\end{pmatrix}{{KR}^{- 1}\left\lbrack {{s\begin{pmatrix}u \\v \\1\end{pmatrix}} - T} \right\rbrack}} + {PRef}} \right)}$

which can be rewritten as:

$\begin{pmatrix}{Lat} \\{Lon} \\{Alt}\end{pmatrix} = {{eccef\_ to}{\_ wgs}84{\left( {P_{3 \times 4}\begin{pmatrix}u \\v \\1\end{pmatrix}} \right).}}$

The set of methods 120 may include the further optional method 124 ofverifying the camera calibration, Preferably, the same first planarchessboard patterned device 140 is used for the verification method 124or several copies of the same first planar chessboard patterned device140 are used.

As shown in FIG. 9 , two additional first planar chessboard patterneddevices 140 are placed at different distances from the camera 101 withinthe camera's FOV. It is preferred that the surroundings have minimalblockage of GPS signals in order to obtain the most accurate globalgeographical coordinates for the test points as “ground-truth”. Themethod 124 includes, as depicted in FIG. 10 , identifying multiplepoints comprising corner points 143 as test points (denoted as 1 to 8 inFIG. 10 ) in each of the additional two first planar chessboardpatterned devices 140. For each test point 143, the global geographicalcoordinates are obtained using, for example, the RTK device or a similarsuch device which can obtain GPS preferably with an error no more than1.5 cm. Furthermore, from the camera image view or image data, the pixelpositions for each of said test points 143 is obtained and the pixelpositions are converted to absolute position data, i.e., to WGS84geographical coordinates. The geographical coordinates obtained for thetest points 143 using the RTK device are compared to the geographicalcoordinates obtained for the test points 143 from the pixel positions inthe camera image view or image data. From the comparison, adetermination can be made of any projection error occurring between thephysically measured geographical coordinates obtained for the testpoints 143 and the geographical coordinates obtained for the test points143 from pixel positions in the camera image view or image data tothereby verify the accuracy of the camera calibration under method 123.The amount of any projection errors may be dependent the positioningaccuracy required for a related application.

Preferably, the test points 143 comprise internal corner points in thetwo additional first planar chessboard patterned devices 140. Cornerpoints are preferred as they are easier to detect but other patterns orshapes such as circular dots are also possible using a suitable imageprocessing algorithm to detect them with subpixel accuracy. The cornerpoints may be detected using the Harris corner detection algorithm orany other suitable corner detection algorithm.

The present invention also provides a method of determining geographicalcoordinates for a point location within an image view of a cameracomprising the steps of, obtaining an image from the camera; selecting apoint location in the image in which the point is lying in the sameplane as the first planar chessboard patterned device; obtainingtwo-dimensional pixel coordinates for said selected point location inthe image; and transforming said two-dimensional pixel coordinates forsaid selected point location in the image into three-dimensionalgeographical coordinates for said selected point location usingprojection data mapping a two-dimensional pixel coordinate system of thecamera image view into a three-dimensional geographical coordinatesystem.

The present invention also provides a camera comprising: a memorystoring machine-readable instructions; and a processor for executing themachine-readable instructions such that, when the processor executes themachine-readable instructions, it configures the camera to implement theaforesaid methods in accordance with the invention.

The apparatus described above may be implemented at least in part insoftware. Those skilled in the art will appreciate that the apparatusdescribed above may be implemented at least in part using generalpurpose computer equipment or using bespoke equipment.

Here, aspects of the methods and apparatuses described herein can beexecuted on any apparatus comprising the communication system. Programaspects of the technology can be thought of as “products” or “articlesof manufacture” typically in the form of executable code and/orassociated data that is carried on or embodied in a type ofmachine-readable medium. “Storage” type media include any or all of thememory of the mobile stations, computers, processors or the like, orassociated modules thereof, such as various semiconductor memories, tapedrives, disk drives, and the like, which may provide storage at any timefor the software programming. All or portions of the software may attunes be communicated through the Internet or various othertelecommunications networks. Such communications, for example, mayenable loading of the software from one computer or processor intoanother computer or processor. Thus, another type of media that may bearthe software elements includes optical, electrical, and electromagneticwaves, such as used across physical interfaces between local devices,through wired and optical landline networks and over various air-links.The physical elements that carry such waves, such as wired or wirelesslinks, optical links, or the like, also may be considered as mediabearing the software. As used herein, unless restricted to tangiblenon-transitory “storage” media, terms such as computer or machine“readable medium” refer to any medium that participates in providinginstructions to a processor for execution.

While the invention has been illustrated and described in detail in thedrawings and foregoing description, the same is to be considered asillustrative and not restrictive in character, it being understood thatonly exemplar embodiments have been shown and described and do not limitthe scope of the invention in any manner. It can be appreciated that anyof the features described herein may be used with any embodiment. Theillustrative embodiments are not exclusive of each other or of otherembodiments not recited herein. Accordingly, the invention also providesembodiments that comprise combinations of one or more of theillustrative embodiments described above. Modifications and variationsof the invention as herein set forth can be made without departing fromthe spirit and scope thereof, and, therefore, only such limitationsshould be imposed as are indicated by the appended claims.

In the claims which follow and in the preceding description of theinvention, except where the context requires otherwise due to expresslanguage or necessary implication, the word “comprise” or variationssuch as “comprises” or “comprising” is used in an inclusive sense, i.e.,to specify the presence of the stated features but not to preclude thepresence or addition of further features in various embodiments of theinvention.

It is to be understood that, if any prior art publication is referred toherein, such reference does not constitute an admission that thepublication forms a part of the common general knowledge in the art.

The invention claimed is:
 1. A method of calibrating a camera comprisingthe steps of: obtaining geographical coordinates of a selected pointlocation within an image view of the camera by placing a first patternedplanar device of known size within the image view of the camera andselecting a point on said first patterned planar device as the selectedpoint location within the image view of the camera, wherein said firstpatterned planar device defines a three-dimensional real-worldcoordinate system of the selected point location; measuring an anglebetween an x-axis of the three-dimensional real-world coordinate systemwith respect to true north, said x-axis passing through said selectedpoint location; and using said obtained geographical coordinates, saidmeasured angle, and projection data derived from characteristics of thecamera to derive modified projection data for transforming atwo-dimensional pixel coordinate system of the camera image view into athree-dimensional geographical coordinate system for point locationswithin the image view of the camera, said projection data comprising aprojection matrix of the camera which transforms the three-dimensionalreal-world coordinate system of said selected point location to saidtwo-dimensional pixel coordinate system of the camera image view, theprojection matrix being derived from a combination of intrinsic cameracharacteristics and extrinsic camera characteristics, and said modifiedprojection data is derived by modifying the projection matrix with afirst rotation matrix based on the measured angle and a second rotationmatrix based on the geographical coordinates of the selected pointlocation to provide an integrated projection matrix; wherein thethree-dimensional geographical coordinate system for point locationswithin the image view of the camera comprises an earth-center, earthfixed (ECEF) coordinate system and the combination of the first rotationmatrix and the second rotation matrix of the integrated projectionmatrix transforms the three-dimensional real-world coordinate system ofsaid selected point location to said ECEF coordinate system; and whereinthe method includes obtaining intrinsic characteristics of the camerafrom a second patterned planar device of known size by imaging saidsecond patterned planar device in multiple different angles and/ordifferent distances within the image view of the camera to obtain anintrinsic characteristics matrix for the camera, the intrinsiccharacteristics matrix comprising a projective transformation of athree-dimensional coordinate system of the camera image view to thetwo-dimensional pixel coordinate system of the camera image view.
 2. Themethod of claim 1 wherein the first rotation matrix transforms thethree-dimensional real-world coordinate system to a North-East-Down(NED) coordinate system and the second rotation matrix transforms theNED coordinate system to the ECEF coordinate system.
 3. The method ofclaim 1, wherein the method includes modifying the integrated projectionmatrix to transform the ECEF coordinate system to a absolute coordinatesystem.
 4. The method of claim 1, wherein the step of obtaininggeographical coordinates for said selected point location comprisesusing a real-time kinematic (RTK) device.
 5. The method of claim 1,wherein the second patterned planar device of known size is smaller thanthe first patterned planar device.
 6. The method of claim 1, wherein themethod includes obtaining extrinsic characteristics of the camera fromthe first patterned planar device to obtain an extrinsic characteristicsmatrix for the camera comprising rotation and translation vectors, theextrinsic characteristics matrix comprising a transformation of thethree-dimensional real-world coordinate system of the selected pointlocation to the three-dimensional coordinate system of the camera imageview.
 7. The method of claim 6, wherein the projection matrix derivedfrom characteristics of the camera comprises a matrix multiplicationproduct of the intrinsic characteristics' matrix and the extrinsiccharacteristics' matrix.
 8. The method of claim 1, wherein the methodincludes obtaining geographical coordinates for multiple point locationson the first patterned planar device using a RTK device; detecting saidmultiple point locations on the first patterned planar device in theimage view of the camera; using the integrated projection matrix todetermine corresponding geographical coordinates for each of saiddetected multiple point locations in the image view of the camera; andcomparing the obtained geographical coordinates using the RTK devicewith the calculated geographical coordinates using the integratedprojection matrix to determine any projection errors between theobtained geographical coordinates using the RTK device and thecalculated geographical coordinates using the integrated projectionmatrix.
 9. The method of claim 8, wherein calibration of the camera isverified or not verified dependent on an amount of any projectionerrors.
 10. The method of claim 8, wherein the step of detecting saidmultiple point locations on the first patterned planar device comprisesdetecting corner point locations on said first patterned planar device.11. The method of claim 8, wherein the step of detecting said multiplepoint locations on the first patterned planar device includes placingsaid first patterned planar device at different distances from thecamera within the image view of the camera and repeating the steps ofclaim 7 for each different distance.
 12. A camera comprising: a memorystoring non-transitory machine-readable instructions; and a processorfor executing the non-transitory machine-readable instructions suchthat, when the processor executes the non-transitory machine-readableinstructions, the non-transitory machine-readable instructions configurethe camera to: receive data comprising geographical coordinates of aselected point location within an image view of the camera by selectinga point on a first patterned planar device of known size placed withinthe image view of the camera, wherein said first patterned planar devicedefines a three-dimensional real-world coordinate system of the selectedpoint location; receive data comprising a measured angle between anx-axis of a three-dimensional real-world coordinate system with respectto true north, said x-axis passing through said selected point location;and using said received data comprising the geographical coordinates,said received data comprising the measured angle, and projection dataderived from characteristics of the camera to derive modified projectiondata for transforming a two-dimensional pixel coordinate system of thecamera image view into a three-dimensional geographical coordinatesystem for point locations within the image view of the camera, saidprojection data comprising a projection matrix of the camera whichtransforms the three-dimensional real-world coordinate system of saidselected point location to said two-dimensional pixel coordinate systemof the camera image view, the projection matrix being derived from acombination of intrinsic camera characteristics and extrinsic cameracharacteristics, and said modified projection data is derived bymodifying the projection matrix with a first rotation matrix based onthe measured angle and a second rotation matrix based on thegeographical coordinates of the selected point location to provide anintegrated projection matrix; wherein the three-dimensional geographicalcoordinate system for point locations within the image view of thecamera comprises an earth-center, earth fixed (ECEF) coordinate systemand the combination of the first rotation matrix and the second rotationmatrix of the integrated projection matrix transforms thethree-dimensional real-world coordinate system of said selected pointlocation to said ECEF coordinate system; and wherein the intrinsiccharacteristics of the camera is obtained from a second patterned planardevice of known size by imaging said second patterned planar device inmultiple different angles and/or different distances within the imageview of the camera to obtain an intrinsic characteristics matrix for thecamera, the intrinsic characteristics matrix comprising a projectivetransformation of a three-dimensional coordinate system of the cameraimage view to the two-dimensional pixel coordinate system of the cameraimage view.